Optimization of Smooth and Strongly Convex Functions

A. Proof of Lemma 1 We need the following lemma that characterizes the property of the extra-gradient descent. Lemma 8 (Lemma 3.1 in (Nemirovski, 2005)). Let Z be a convex compact set in Euclidean space E with inner product 〈·, ·〉, let ‖ · ‖ be a norm on E and ‖ · ‖∗ be its dual norm, and let ω(z) : Z 7→ R be a α-strongly convex function with respect to ‖ · ‖. The Bregman distance associated with ω for points z,w ∈ Z is defined as Bω(z,w) = ω(z)− ω(w)− 〈z−w,∇ω(w)〉. Let U be a convex and closed subset of Z, and let z− ∈ Z, let ξ,η ∈ E, and let γ > 0. Consider the points